Skip to content

fix(structured): reject a non-array anyOf/oneOf as a schema error - #1364

Merged
NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
aashish254:fix-structured-union-nonarray
Oct 29, 2026
Merged

NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
aashish254:fix-structured-union-nonarray

Conversation

@aashish254

@aashish254 aashish254 commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

What

laya/structured.py::_field iterated a property's anyOf/oneOf union without first checking it is an array, while the allOf branch immediately above guards its wrapper with isinstance(wrapped, list) and _enum_field guards its enum the same way. A property whose union is a JSON number — or a boolean, which subclasses int — fell straight into the branch comprehension and leaked TypeError; a truthy non-array string/object iterated to zero branches and reached the union-arity refusal ... got 0, which blamed an arity the request never had.

Now the union is unwrapped only when isinstance(union, list); any other shape falls to the existing terminal SchemaError that names the path.

Per @NandhaKishorM's review, the same refusal is carried across the ports so all three agree:

  • Java (Decisions.java): the union unwrap is gated on union instanceof List, and the "'anyOf' must be an array of branches'" exception is removed so a number/boolean/other non-array now falls to the terminal unsupported schema like the reference.
  • TypeScript (structured.ts): already gates on Array.isArray(union) and falls through to unsupported schema — verified, no change needed.
  • The fixture laya-java/fixtures/structured.json is regenerated with gen_fixtures.py; exactly the two truthy-non-array rows change.

Why

The module's contract is a SchemaError naming the offending path. Every sibling raise honours it — {'enum': 5} → 'enum' must be an array of values, got int and {'allOf': 5} → unsupported schema {'allOf': 5}. This one leak was the odd case out, and the old wording was a cross-port contract that Java reproduced deliberately.

The maintainer asked whether the number case should keep a third wording; it now joins unsupported schema, so a non-array union has one message across all three ports.

How it was verified

# Python: non-array union -> unsupported schema naming the path; the genuine
# one-non-null-branch list unions and the list-with-only-null union are unchanged.
python -c "from laya.structured import questions_from_json_schema as q; q({'type':'object','properties':{'x':{'anyOf':3}}})"
# -> properties.x: unsupported schema {'anyOf': 3}   (was TypeError)
#    {'anyOf':[{'type':'null'}]} still -> ... supported, got 0

python tests/test_structured.py         # 196 -> 206 checks, 0 failed
python tests/test_structured_docs.py    # 117 checks passed (no new message literal)
python tests/test_structured_api.py     # 38 passed, 0 failed

# Java: DecisionsTest is fixture-driven, so the regenerated fixture is the oracle.
JAVA_HOME=/opt/homebrew/opt/openjdk@17 ./gradlew --no-daemon test --tests '*DecisionsTest'
# -> 99 tests, 0 failures, 0 errors, 0 skipped (BUILD SUCCESSFUL, JDK 17)

# Fixture regen: --check was stale, regenerated, now clean.
python laya-java/scripts/gen_fixtures.py --check --only structured
# -> "1 family matches laya" after regen; exactly 2 changed rows, both truthy-non-array.

# Mutation arm: base Decisions.java against the new fixture fails on EXACTLY those 2
# union rows; restoring the fix -> 99/0 green again.

# TypeScript: runtime-probed the three inputs -> all throw "unsupported schema";
# structured.test.ts stays green (57 tests) with no source change.

Checklist

  • Focused on one change
  • Rebased on latest main
  • Tests pass locally
  • Docs updated when public API changed (no public API change; docs/structured.md rejection table already covers the unsupported schema message this path raises — verified by tests/test_structured_docs.py)

@NandhaKishorM

Copy link
Copy Markdown
Owner

Holding this one, and I want to be precise about why, because the change itself is an improvement and the reason is not about your code.

The old message for {"anyOf": "abc"} was properties.x: only 'Optional[...]' unions (one non-null branch) are supported, got 0, which describes a union arity problem when the real fault is that anyOf is not an array at all. Your unsupported schema {'anyOf': 'abc'} says the true thing. No argument there.

The problem is that the old wording is a cross-port contract, and two ports reproduce it deliberately. laya-java/.../Decisions.java carries this comment above the exact branch:

A FALSY anyOf falls through to oneOf, because the reference spells this prop.get("anyOf") or prop.get("oneOf"). Falsy is Python's, not "not a list" [...] while a NON-EMPTY string or object is truthy and does NOT, it is iterated, yields no mapping branch, and lands on the got 0 refusal below. Reproducing only the empty-list case sent a truthy {"anyOf": "abc"} through to oneOf and, with no oneOf to find, out to "unsupported schema".

So someone already walked into the exact case you are fixing, from the other direction, and wrote the port to match Python's got 0. laya-ts/src/structured.ts:215 has the same path.

Measured rather than assumed: with this PR on the branch, gen_fixtures.py --check reports structured.json is stale, and regenerating produces exactly two changed rows, both yours:

-      "error": "properties.x: only 'Optional[...]' unions (one non-null branch) are supported, got 0"
+      "error": "properties.x: unsupported schema {'anyOf': 'abc'}"
-      "error": "properties.x: only 'Optional[...]' unions (one non-null branch) are supported, got 0"
+      "error": "properties.x: unsupported schema {'anyOf': {'a': 1}, 'oneOf': [{'type': 'boolean'}, {'type': 'null'}]}"

With this PR removed and the other eight merged, every Java gate is clean again: 20 families match, UnicodeTables.java matches CPython 15.0.0, LanguageTables.java matches laya/lang.py. So the drift is this change alone, and the Java structured test would fail on the regenerated fixture because the port still emits the old string.

What would make this mergeable, and I think it is worth doing:

  1. The same refusal in laya-java/.../Decisions.java and laya-ts/src/structured.ts, so all three agree. Only those two implement schema decisions; Rust, .NET and Go have no equivalent path, so they need nothing.
  2. python laya-java/scripts/gen_fixtures.py committed with it, which is the two rows above.
  3. Worth deciding explicitly: {"anyOf": 5} currently raises 'anyOf' must be an array of branches, got int because a number is not iterable, and that is recorded in Decisions.java as the one deliberate exception to the message-for-message rule. If a non-array is now unsupported schema, the number case probably wants to join it rather than keep a third wording.

I can carry it across the ports myself if you would rather not, but I would be writing Java I cannot run locally, so it is better from you. Either way the Python half is right and I am not asking you to change it.

The other eight PRs in this batch merged clean and are going out in the next release. This is the only one held.

…t a TypeError

The allOf branch guards its wrapper's shape before iterating it; anyOf/oneOf
did not, so a schema whose union is a JSON number (or a bool, which subclasses
int) fell straight into the branch comprehension and leaked
`TypeError: 'int' object is not iterable` instead of the path-naming SchemaError
the module raises for every other malformed property -- including the sibling
non-array `enum` case.

A non-array union is not a recognised optional wrapper, so the guard now lets it
fall through to the terminal "unsupported schema" rejection, which names the
field path. Well-formed one-non-null-branch unions still unwrap unchanged.

Adds the rejection cases and a positive control to tests/test_structured.py
(196 -> 206 checks; the structured docs/api suites stay green).
…hing Python

laya/structured.py::_field now guards the union with isinstance(union, list),
so a truthy non-array anyOf/oneOf (string, object, number, boolean) falls to
the terminal "unsupported schema" rejection instead of leaking a TypeError or
reaching the "got 0" union-arity refusal. Decisions.java reproduced the old
behaviour: it iterated a truthy non-array to zero branches ("got 0") and raised
"'must be an array of branches'" for non-iterables -- the one recorded exception
to the message-for-message rule.

Gate the unwrap on `union instanceof List` and let any other shape fall past the
block, so all three message-producing ports (Python, Java, and TypeScript, which
already gated on Array.isArray) agree. Regenerate the fixture with gen_fixtures.py:
exactly two refusal rows change, both the truthy-non-array anyOf cases; the genuine
list-union "got 0" (union-of-null-only) is untouched.
@aashish254
aashish254 force-pushed the fix-structured-union-nonarray branch from e0c30ee to 6df415f Compare October 25, 2026 16:31
@aashish254

Copy link
Copy Markdown
Contributor Author

Thanks for the precise read — that's exactly right, and I've carried it across the ports rather than leave the drift for you to write blind in Java.

Short answer: the fix is now on the branch across all three ports, rebased on current main, with the fixture regenerated. The Java gate should go green on this push.

What changed since your review:

  1. Java (Decisions.java) — the union unwrap is now gated on union instanceof List, and the "'anyOf' must be an array of branches'" branch is removed, so a truthy non-array (string/object/number/boolean) falls to the terminal unsupported schema like the reference. The falsy anyOf-to-oneOf fallthrough is untouched.
  2. TypeScript (structured.ts) — no change needed. It already gates on Array.isArray(union) and falls through to unsupported schema. Verified at runtime: {"anyOf":"abc"}, {"anyOf":{"a":1},...} and {"anyOf":5} all throw unsupported schema, and structured.test.ts stays green (57 tests).
  3. Fixture — regenerated with gen_fixtures.py. Exactly your two rows change; gen_fixtures.py --check --only structured is clean after it, and the genuine list-with-only-null union still records got 0.

On the number case you flagged: I went with unifying it rather than keeping a third wording — {"anyOf": 5} now gives unsupported schema in every port. So I did not add a {"anyOf": 5} row to the fixture, which keeps the diff at the two rows you measured; if you'd rather pin the number case explicitly as its own refusal row, say so and I'll add it.

Witness: DecisionsTest is fixture-driven, so it exercises this end-to-end — JAVA_HOME=<openjdk@17> ./gradlew test --tests '*DecisionsTest' gives 99 tests, 0 failures/errors/skips. Mutation arm: the base Decisions.java against the new fixture fails on exactly those two union rows; restoring the fix returns it to 99/0.

Appreciate you offering to write the Java yourself — glad to have it from a machine that can actually run the gate.

NandhaKishorM added a commit that referenced this pull request Oct 25, 2026
8 PRs. Rust and .NET batch prediction and router batch APIs (#1358,
#1359), the min_confidence abstention gate and jev_confidence in both
ports (#1362, #1363), and four inputs refused where they enter instead
of failing later (#1365, #1366, #1367, #1368).

Verified before tagging: all 92 suites pass, 612 tests. ruff clean on
both lists. Java fixture, Unicode table and language table gates clean;
three Go tables_gen.go match. Version agrees across pyproject.toml,
laya.__version__ and nix/package.nix. README carries 0.4.16 only.

Security audit of the merge diff, 180330 added lines over 349 files:
zero dynamic exec, zero network calls, zero URLs, zero base64 or
obfuscation, zero secrets in the diff or the tracked tree, zero
laya-max boundary hits, no writes outside the test temp dir, the
dependency surface untouched, no npm install hooks, no binaries, every
new file 100644, and 20 of 20 actions SHA-pinned with codeql-action
init, autobuild and analyze all on 4.39.1.

Held from this release: #1364 (correct in Python but it moves a
cross-port contract Decisions.java reproduces deliberately and
structured.ts shares), #1324 (real table regression), and seven
dependency and action bumps for a pass of their own.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

@rahul05ranjan rahul05ranjan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran the new rejection block against this branch's laya/structured.py (harness with the module's own helpers, no other deps): all pass — non-array anyOf/oneOf values (3, 1.5, True, False) raise SchemaError naming the field and echoing the schema, {"anyOf": 3} produces exactly properties.x: unsupported schema {'anyOf': 3}, a real one-non-null-branch union still unwraps to noul, and the pre-existing rejections (free string, empty properties) are unchanged. A null-only branch list still refuses with the arity message, and a falsy anyOf ([]) still falls through to oneOf — the Python truthiness semantics the class note documents survive the new guard.

The Java side mirrors it correctly: union instanceof List matches isinstance(union, list) in the reference, and the fixture updates show Java now reporting the same "unsupported schema" text (including the Python-repr formatting) instead of the old arity complaint — which was misleading anyway, since the request never had branches to count. Dropping the old SchemaException for non-iterables removes the one recorded exception to the message-for-message rule; a uniform rejection is the better trade.

Trace-only note, since I can't build the Java side here: in Decisions.java a truthy-but-not-a-List anyOf (e.g. a non-empty Map) no longer falls through to oneOf and can't reach the branch loop at all. I read that as intentional and it matches the Python ({"anyOf": {...}} → terminal rejection, per the fixture), just flagging it in case a Map-typed anyOf was ever expected to be unwrapped.

NandhaKishorM added a commit that referenced this pull request Oct 28, 2026
24 PRs. Vietnamese kept off the English checkpoint (#1392), deciding-window
attribution preserved through end hooks (#1369), RemoteRouter identifying
itself with a configurable User-Agent (#1396), ten more inputs refused where
they enter (#1372, #1394, #1397, #1398, #1400, #1401, #1402, #1403, #1404,
#1405, #1406), per-language temperatures and per-call budgets in .NET and
Rust (#1370, #1371), .NET calibration files (#1384), the Go router lifecycle,
SHA-256 pins and routed inference (#1374, #1379, #1395), a C++ foundation
(#1399, #1407), and parity-lane sharding that replaces the budget headroom
added for 0.4.16 (#1373, #1375).

Verified before tagging: all 92 suites pass, 640 tests, up from 612. ruff
clean on both lists. laya-ts 1251 tests across 44 files, tsc --noEmit clean.
Every generated-artifact gate clean on CPython 3.12 / unicodedata 15.0.0:
Java 20 families match laya (9 unverified), UnicodeTables matches CPython
15.0.0, LanguageTables matches laya/lang.py, three Go tables_gen.go match,
both laya-cpp .inc files match. Version agrees across pyproject.toml,
laya.__version__ and nix/package.nix. README carries 0.4.17 only.

Security audit of the merge diff, 362766 added lines over 480 files: zero
obfuscation, zero secrets, zero boundary hits, zero binaries, no mode or
symlink changes, 20 of 20 actions SHA-pinned including the new cpp.yml, npm
install hooks unchanged, and no third-party host in any added line
(example.com 90, github.com 2, download.pytorch.org 1, docs.rs 1). The 12
dynamic-exec hits are the C++ test harness invoking its own checker through
sys.executable, the Go sample generator reading laya's own example by AST at
development time, and one comment in rust.yml.

Held: #1364 (unchanged, it moves a cross-port contract), #1381 (a #[doc(hidden)]
but pub Rust hook that would ship as public API able to replace the forward
pass), #1386 (route_batch switched from self.route to self._route, silently
bypassing subclass overrides the docstring promises), and nine stacked port
PRs that conflict with each other and need linearizing (#1377, #1378, #1380,
#1382, #1383, #1385, #1387, #1388, #1389, #1390).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@NandhaKishorM

Copy link
Copy Markdown
Owner

You did everything I asked and then some, and the mutation arm is the part that makes this reviewable rather than just assertable. I re-ran it all rather than taking it on trust, and every claim holds:

  • Merges clean onto current main (fe9a6c35), no conflicts.
  • gen_fixtures.py --check: 20 family/families match laya. structured.json is no longer stale, which was the whole blocker.
  • The fixture diff is exactly the two rows I measured, no third row and no collateral:
-      "error": "properties.x: only 'Optional[...]' unions (one non-null branch) are supported, got 0"
+      "error": "properties.x: unsupported schema {'anyOf': 'abc'}"
-      "error": "properties.x: only 'Optional[...]' unions (one non-null branch) are supported, got 0"
+      "error": "properties.x: unsupported schema {'anyOf': {'a': 1}, 'oneOf': [{'type': 'boolean'}, {'type': 'null'}]}"
  • Python suite: 640 passed, 92 suites.
  • laya-ts structured.test.ts: 57 passed, confirming your read that TypeScript already gated on Array.isArray(union) and needed nothing.

Gating the Java unwrap on union instanceof List while leaving the falsy anyOf to oneOf fallthrough untouched is exactly right, since that fallthrough is Python's truthiness and not a list check. Those are two different questions in one expression and keeping them apart is the only way the port stays faithful.

On the number case: unifying it was the right call and I am glad you took that rather than preserving a third wording. Three spellings for one fault is how a cross-port contract becomes unmaintainable. Please do add the {"anyOf": 5} row to the fixture though. You are correct that it keeps the diff minimal to leave it out, but the reason it had a deliberate exception recorded in Decisions.java is that someone previously got it wrong, and an unpinned case is free to drift back. One extra row is cheap insurance against exactly the regression that produced the comment you quoted.

Timing, so you are not left guessing. 0.4.17 is already built, verified and in flight: tagged off a candidate with all nine lanes green at job level, so I am not going to reopen it to add a PR, because that would discard the verification. This goes in first thing in 0.4.18, as soon as you push the number row. If you would rather ship it as is, say so and I will merge without that row, but my preference is to pin it.

Thanks for the patience across two releases on this one. The reason it was held was never your Python change, which was right from the first push.

@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged in 0.4.18. Thank you for carrying this across the ports rather than fixing Python alone.

isinstance(union, list) is the right guard, and it is the same shape the allOf branch above it already uses, which is what makes the two read consistently now. I verified the reference behaviour rather than reading the diff:

{'anyOf': 5}     -> SchemaError: properties.x: unsupported schema {'anyOf': 5}
{'oneOf': True}  -> SchemaError: properties.x: unsupported schema {'oneOf': True}

Your Python tests cover (3, 1.5, True, False) with an exact-message assertion, which is the thorough version of this.

I added one commit on top rather than send it back for a third round. Two things:

The cross-port fixture pinned only the shapes that were iterable, a truthy string and a truthy object. A truthy number and a truthy boolean are the ones that leaked TypeError: 'int' object is not iterable, and bool subclasses int so true leaked the same way. Neither had a fixture row, so a port could regress to the old custom message with every fixture still green. I added truthy-non-array-anyof-number and truthy-non-array-oneof-boolean to gen_fixtures.py and regenerated.

The generator comment above those cases described the behaviour your change replaced, and went further: it warned that a port "testing is it a list gets wrong in the opposite direction" by refusing with "unsupported schema". That is now exactly what the reference does, so the comment argued against the implementation it documents. Rewritten to name the four shapes and why each stays pinned.

Worth knowing for next time, since it is not obvious from the diff: laya-java/fixtures/structured.json is generated, so a new case goes in gen_fixtures.py and the fixture is regenerated on the pinned toolchain, not hand-edited.

Verified before merging: 706 tests across 93 suites, gen_fixtures.py --check reports 20 families matching, and the two new rows carry exactly the messages the reference raises.

@NandhaKishorM
NandhaKishorM merged commit accfe80 into NandhaKishorM:main Oct 29, 2026
52 checks passed
NandhaKishorM added a commit that referenced this pull request Oct 29, 2026
… iterable

#1364 replaced the iterate-and-count path with `isinstance(union, list)`, which is
the right guard, but the cross-port fixture only pinned the two shapes that WERE
iterable: a truthy string and a truthy object. A truthy number and a truthy boolean
are the shapes that leaked `TypeError: 'int' object is not iterable` (`bool`
subclasses `int`), and nothing in the Java fixture held either, so a port could
regress to the old custom message without a fixture failing.

The generator comment described the behaviour #1364 replaced, and argued against
testing "is it a list" when that is now exactly what the reference does. Rewritten
to say what the four shapes are and why each is pinned.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
NandhaKishorM added a commit that referenced this pull request Oct 29, 2026
Twelve pull requests: #1364, #1409, #1410, #1412, #1413, #1414, #1415, #1416,
#1417, #1418, #1419, #1420.

Test suites: 706 passing across 93 suites, up from 640 across 92. laya-cpp
registers 423 ctest tests, up from 213, and its MIN_TESTS floor is raised to match.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants